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ABSTRACT 



With approximately 50 percent of school administrators 



retiring over the next 10 years, school leader preparation programs are under 
pressure to admit adequate numbers of candidates to justify their costs and 
continued existence. Candidate recruitment and selection suffer, however, 
from ongoing practical problems including the use of grade-point averages, 
recommendations/references. Graduate Record Examination scores, writing 
samples, and personal interviews as primary criteria for assessing 
candidates' merits. Consequences are low standards of admission, lowered 
quality of instruction, diminished status of educational administration 
programs in the eyes of the public, and candidates' receiving credentials 
just by taking requisite courses. Auditioning candidates in much the same way 
as is done in the performing arts is recommended as a behavior-based way of 
evaluating how potential school leaders perform in real-life situations. Two 
scripted scenarios are described in which a candidate interacts with an irate 
parent over the phone, and another interacts personally with a concerned 
parent group. Assessment forms provide information on how effectively each 
scenario was handled in ways not necessarily addressed by traditional 
criteria. It is hoped that by adding the audition to school -leader selection 
criteria, the selection process itself will be improved. (Contains 21 
references and 4 tables.) (RT) 
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There is some evidence that a more proactive stance to the selection of potential 
school leaders in our university preparation programs has taken place over the last decade 
(Murphy, 1999). In rare cases, the selective measures have expanded beyond the 
traditional to include assessment center activities and interviews. Upon a more thorough 
inspection however, the majority of universities still rely primarily on the traditional 
measures of Graduate Record Examination scores, letters of recommendation, and grade 
point averages (Creighton & Jones, 2001). 

University preparation programs are under pressure to take in adequate numbers 
of candidates to justify the program’s costs and existence (Sarason, 1999). This pressure 
results in admitting individuals of borderline quality, using the usual criteria. With the 
projection of approximately 50% of current school administrators leaving our schools due 
to retirement in the next decade (Harris, 2000), it is more than crucial that we attract high 
quality applicants to school administration preparation programs. The selection process 
can and must be improved. This article points to the importance of the use of “behavior- 
based measurements” to select candidates for educational administration preparation 
programs. Specifically, the recommendation is made to borrow a strategy used in the 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



V. 



The Problem 



The field of educational leadership has long been criticized for the ways in which 
men and women are prepared for school leadership positions (Murphy, 1999). In 1960, 
the American Association of School Administrators characterized the preparation of 
school superintendents and principals as dismal. Later Farquhar and Piele (1972) 
described university-based preparation programs as “dysfunctional structural 
incrementalism’Xp. 17). As recently as 1990, Pitner discussed the “zombie programs” (p. 
131) in educational administration. The most recent research in educational leadership 
reform warns that no aspect of our leadership preparation programs is more damaging 
than our reputation for being a haven for mediocre candidates. After 40 years of 
continued alerts, the field of educational administration still has serious problems in the 
ways candidates are recruited (if at all) and especially the ways candidates are selected 
into educational leadership preparation programs. 

In 1991, the University Council for Educational Administration (UCEA) initiated 
a national study of recruitment and selection processes in UCEA member institutions. 

The focus of the study was to determine the existing practices relative to student 
recruitment and selection and to find what strategies were being implemented to attract 
minorities and women and other talented populations to educational administration 
preparation programs (Murphy, 1999). Of the then 50 UCEA member institutions, 40 
participated. 

The leading sources of evidence for determining program admission for students 
were: (a) grade point averages, (b) recommendations/references, (c) the Graduate Record 
Examination score, (d) the use of writing samples, (e) personal interviews, and (f) Millers 
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Analogies Test results. When participants were asked to provide their best judgement as 
to the weight given to each item in the selection process, they reported the GRE, grade 
point averages, recommendations, writing samples, and the personal interview. 

Murphy (1999) reported on the changes in preparation programs from 1989 to 

1996 by comparing survey responses from chairpersons of educational administration 
0 

departments in 54 University Council of Educational Administration (UCEA) 
institutions. There was evidence of a “renewed commitment to employ selection tools in 
a thoughtful fashion in the service of collecting and assessing information about 
candidates” (p.l78). Department chairs reported a use of personal interviews, examples 
of written work, and documentation of leadership experiences in the selection of 
candidates. Perhaps some limitations (or assumptions) of this study relate to whether or 
not accurate information was reported by department chairs, the relatively small sample 
size, and the somewhat bias selection of UCEA member universities. 

In an effort to expand the research base, Creighton and Jones (2001) reviewed 
educational administration selection requirements at universities in U. S. and Canada. 
The results are not encouraging. Of the 350 university programs reviewed, only 7% (26) 
require the interview as part of the selection criteria, with an additional 2% (7) reporting 
the potential use of interviews, depending on quality and level of other criteria. The data 
collected revealed a continued dependence on Graduate Record Examination scores, 
undergraduate grade point average, and letters of recommendation as the most frequently 
used criteria in the selection of candidates for educational administration programs. 

Little evidence surfaced revealing the existence of behavior-based measures or 
consideration of documentation of leadership experiences. 
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The problem relates to how we recruit and select our school administrators for our 
university preparation programs. The most common recruitment/selection strategies 
accept future administrators into programs under three criteria: (a) grade point average, 

(b) Graduate Record Examination Scores (GRE), and (c) letters of recommendations with 
little evidence of consistent standards of acceptance (AACTE, 1994). Recent research 
has identified the criteria weighted most heavily in the selection of future administrators 
as the GRE score (Norton, 1994). 

A secondary problem is whether the current admissions standards are sufficiently 
high to attract the “brightest and most capable candidates” to our programs (Norton, 

1994, p.41). The UCEA study found on average, an undergraduate grade point average 
of 2.8, a graduate grade point average of 3.3, and a GRE score of only 950 (verbal and 
quantitative combined) being required for program admission. Present studies (Creighton 
& Jones, 2001) find these same averages in place today, ten years later. A particularly 
troubling finding reveals many university preparation programs have implemented a 
waiver of one or more of the requirements. For example, if a candidate has a higher than 
required grade point average, but scored low on the GRE, he/she is admitted on the basis 
of meeting one of the two requirements. Often, they are admitted “conditionally” with 
the condition being a completion of 6-9 hours of course work with at least a B average. 
The “haunting question” returns: What is the correlation between a student maintaining a 
B average and his/her administrative potential? 

Data published in the GRE Guide reveal that students entering educational 
administration preparation programs from 1989 - 1992 attained GRE scores ranking 
fourth (4*) from the bottom when compared with 41 graduate fields. Compared with ten 



specific graduate education majors, educational administration ranked third from the 
bottom. The most recent GRE Guide (1999) containing data collected from over 2,775 
educational administration majors during the years of 1996 - 1999, reports their scores 
ranking eighteenth (IS'*') from the bottom of 289 graduate fields and compared with the 
ten specific graduate education majors, educational administration ranked second (2"**) 
from the bottom. 

The present use of Graduate Record Examination scores, undergraduate and 
graduate grade point averages, and letters of recommendation for the selection of 
candidates for school administration in isolation are not in question - they have their 
place - but when they are the sole basis for selection, they are and have been found to be 
only partially effective (Sarason, 1999). Although various selection criteria are used, the 
dominant one is grade point average; only limited attention is given to factors associated 
directly with administrative potential (Creighton & Jones, 2001). 

The lack of rigorous recruitment and selection procedures has several potentially 
negative effects: 

1. Weak recruitment and selection processes lower the quality of 
instruction offered, since courses are often geared to the background 
and intelligence of the students. 

2. Easy entry diminishes the status of educational administration 
programs in the eyes of the public. 

3. The candidates themselves realize that anyone can get the credential if 
he/she keeps paying for credits. (Cooper & Boyd, 1987) 
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4. Low standards of admission permit (and encourage) enrollment of 
candidates interested only in a master’s degree in education with little 
intent of vigorously pursuing an administrative position. (Creighton & 
Jones, 2001) 

Increasing the level of selection will result in a higher quality of administrator 
prepared by our universities, thus providing better principals and superintendents for the 
public schools in the nation. Many studies point to the importance of quality leaders 
(Fullan, 1994; Sarason, 1995; Schlechty, 1997; Sergiovani, 1996), and support exists for 
the stance that no amount of educational reform or restructuring will occur without strong 
effective school leadership. The prediction that as many as 50% of school administrators 
in the nation will leave their positions over the next five years due to retirement, 
relocation, and career change translates to the potential of replacing approximately half of 
the education leaders in our country in a relatively short period of time. Emphasizing that 
strong effective leaders have a direct effect on student achievement and on organizational 
change, the impact and potential for implementing major educational reform in this 
country is immense. A window of opportunity exists to radically improve the quality of 
education. Improving the recruitment and selection of our school administrators must 
become a high priority in our university preparation programs. How will we respond to 
this need for numbers? Will we address the supply and demand with current selection 
processes, or will we focus on the opportunity to improve quality and effectiveness in 
education? If the latter, we must be certain that our university recruitment and selection 
procedures are rigorous, effective, and cost efficient. 
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Auditions Versus Interviews 



Several have suggested the need to view leadership as a performing art rather than 
a set of skills, competencies, and knowledge (Achilles & Mitchel, 2001; Sarason, 1999; 
Vaill, 1989). Graduate schools in the performing arts certainly require traditional criteria 
such as previous grades, test scores, and interviews, but no school accepts a student 
without an audition - they want to see how the student performs. Auditions are not 
interviews. They are samples of behavior displayed under real-life situations. We cannot 
afford to admit individuals into preparation programs without some form of audition 
which will give us a basis for assessing how they will interact with children, parents, and 
teachers in situations calling for inventiveness, spontaneity, and sensitivity (Sarason, 
1999). 

Interviews (and to some extent assessment center activities) usually focus on a 
predetermined set of questions or hypothetical situations and only allow the candidate to 
tell how and why and what he/she would do to handle the situation. Auditions, on the 
other hand, immerse the candidate into the real-life environment and require the 
candidate to actually demonstrate (perform) a behavior. What a person says he/she will 
do is far from correlated with what a person actually will do when confronted with the 
situation (Sarason, 1999). As in the performing arts, auditions used in administrative 
recruitment and selection do not require a perfect performance - we realize we are 
observing untrained individuals. But, does the candidate display the “qualities” 
necessary for performing effectively with children, parents, and teachers? 

Though the interview is considered an important administrator screening tool 
(Baltzell & Dentler, 1983; Bryant, 1978; Pounder & Young, 1996; Schmitt & Cohen, 
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1990), candidate interviews are often unstructured, informal, and inconsistent. Even 
when highly structured selection guidelines are followed, it is difficult to predict future 
job performance, unless the selection system focuses on behavior-based measures. 

Auditions provide an opportunity to observe the candidate in a realistic school 
environment. For example, auditions can focus on: (1) an unscheduled meeting with a 
hostile parent, (2) talking with a student contemplating suicide, (3) taking a phone call 
from an irate parent, or (4) presenting a report of declining standardized test scores to the 
board of education. More specific examples of auditions and their use will be reported 
later in this article. 

Weaknesses of the Interview 

Research has shown the interview to contain several weaknesses (http://www.hr- 
guide.com/data/G3 1 l.htm) . Selection decisions based upon the interview reveal 
disproportionate rates of selection between minority and non-minority members. 
Decisions tend to be made within the first few minutes of the interview with the 
remainder of the interview used to validate or justify the original decision. 

In a recent study, Salzman & Denner (1998) reported the traditional interview 
indicators tell us little about the affective, moral, or ethical dispositions of the students 
who are admitted into education programs. Goodlad (1990) suggests that “the 
responsible group of academic and clinical faculty members must seek out and select 
those candidates who reveal an initial commitment to the moral, ethical, and 
enculturating responsibilities to be assumed” (pp. 282-283). While some institutions 
have begun to use interviews as part of their admissions procedures, it is not clearly the 
case that the questions asked and the criteria are structured. Too often, interviews merely 
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contribute impressionistic data about a candidate’s desire to enter the program rather than 
systematic evidence that admission standards have been met. Lacking such standards, 
interviews become at best “procedures for screening students out rather than selecting 
them into the program” (Salzman & Denner, 1998, p. 4). 

In many cases, the interview questions are not job related resulting in the 
potential for racial and gender stereotypes in the interview process. In addition, evidence 
reveals a tendency to give negative information more weight than positive information. 
For the interview process to be effective, interviewers must: (a) avoid asking questions 
unrelated to the job, (b) avoid making quick decisions about the applicant, (c) avoid 
stereotyping applicants, (d) avoid giving too much weight to a few characteristics, and (e) 
maintain consistency in the questions asked. Because of these confounding variables, 
little evidence of the validity exists of the interview used for a selection procedure. 
Validity of Selection Strategies 

Schmitt & Cohen (1990) reported the validity coefficients of various selection 
techniques used to select managerial and professional employees, as shown in Table 1. 
Table 1 Validity Coefficients for Management Selection Techniques 

Selection Technique Validity Coefficient 



Interviews 


.095 


Biographical Information 


-.011 


Mental Ability Measured by Paper/Pencil 


.186 


Paper/Pencil Personality Inventory 


.126 


Assessment Center Activities 


.325 


Supervisory /Peer Ratings 


.441 





These results suggest that selection systems for managers that focus on behavior- 
based measures result in the highest validity. Though university preparation programs 
use assessment center activities on a limited basis, it is rare to receive appropriate 
supervisory/peer rating on our candidates, since most are not practicing administrators. 
Any supervisory rating (letters of recommendations) deal with the candidates 
performance as a classroom teacher or at best, the supervisor’s prediction of a candidate’s 
leadership qualities. Only the assessment center activities validity coefficient from the 
Schmitt study is significant to educational administration selection programs. However, 
along with work done by Ponder (1989) and Ponder and Young (1996), evidence reveals 
the following: 

1. Administrator selection practices are often unsystematic and unstructured, rely 
heavily on interview impressions, and would be expected to have weak 
predictive powers. 

2. Selection techniques should be designed to more nearly capture actual or 
simulated job behaviors. 

3. Reliable and valid assessment of actual or simulated job behaviors may more 
readily represent genuine occupational qualifications than many of the 
traditional measures used in candidate screening. 

Auditioning in Educational Administrative Preparation Programs: An Application 

Auditions were a required component of the principal preparation program at 
Idaho State University from Fall semester 1997 through Spring semester 2000. During 
that time a total of 225 masters students preparing for the principalship went through an 
audition process. Though the process was used in this case as a way to identify strengths 
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and weaknesses of enrolled candidates, the intent of this paper is to suggest and 
recommend that auditions be implemented as an additional strategy for selection of 
candidates before they are accepted into preparation programs. 

Methodology . As part of the course entitled, The Principalship, students were 
required to attend a full-day (Saturday) session consisting of a variety of auditioning 
activities. No individual names were used during the day; students began the day as a 
hypothetical principal and identified only by a number (e.g., A-1, A-2, A-3, etc.). 
Activities were scheduled in several rooms and individual offices, and in a manner 
whereby each student progressed through the activities during the day. Evaluators and 
judges were practicing teachers, principals, superintendents, university professors, and 
students. Each candidate’s performance was judged and reviewed by at least 3 
evaluators. 

Approximately one week after completing the interview activities, principal 
candidates received an assessment of each activity based on the feedback of the 
evaluators. The purpose of the assessment report was to provide candidates with 
important information which could be used to identify individual areas of improvement. 
For instance, if a candidate received a weak assessment in the area of communications 
with parents, the candidate hopefully might use that information to select an additional 
elective course or perhaps attend a specific workshop dealing with effective 
communication skills. 

The following examples of audition activities used during 1997-2000 comes from 
the ERIC Clearinghouse on Educational Management’s Project-Based Learning Project 
(Bridges, 1994). 




11 



12 



Specific audition activities included; (a) presenting a principal’s report to a board 
of education, (b) talking with a student who reported physical contact by a classroom 
teacher, (c) handling a drunk and irate father wanting to see his son, and (d) presenting a 
supplies and materials budget reduction at a teachers’ meeting. Two others are provided 
here along with procedures and assessment reports. 

Audition #1. Irate Parent Phone Call. Students answered a phone call from an 
angry parent at scheduled times in a location previously announced (e.g., private office). 
Evaluators performed the role of the irate parent making the call from another office, and 
were accompanied by two additional evaluators. The parent script follows: 

You are Mrs. Robert Wills and want your son transferred out of 
Mrs. Jones’ room. 

Your son is Joe, an 8* grader. 

You have aspirations for him to attend college and it is important 
that he do well in school and learn. 

Your son Joe is not learning anything in Mrs. Jones’ class - it 
is a total waste of time. 

You want your son transferred to Mrs. Johnson; Joe likes her 
and learned a lot from her when he had her last year. 

There must be other parents who feel the same way - 

have you heard from any of them? What are you going to do about it? 

Indicate you don’t care if the principal denies the request. 

Your son Joe is a special case, and the principal better reconsider. 

If he doesn’t transfer Joe, he will hear from you. 
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You mean business! 



And you intend to talk with the Board President and Superintendent. 

The candidate’s responses are listened to on a speaker phone allowing the 
evaluators to hear the conversation. Both the irate parent and the candidate are in the 
privacy of individual offices and the candidate is in no way identified by name. How is 
the candidate’s audition evaluated? 

The evaluation form consists of administrative constructs and descriptors taken 
from NAESP, NASSP, and NPBEA assessment documents used in professional 
assessment center simulations. A completed assessment form used in the irate parent 
phone call audition is shown in Table 2. 

The important issue, as with auditions in the performing arts, is not necessarily a 
perfect performance, but whether or not the candidate possesses the important qualities 
required in the handling of an irate parent phone call. You will notice that additional 
comments focus on constructive criticism and allow the candidate to reflect on areas of 
improvement. The assessment obviously involves subjective decisions by the evaluators. 
These decisions however, are based on many years of experience of practicing teachers, 
administrators, and university faculty. The point is that our profession is currently 
depending excessively on objective data (test scores and grade averages) when accepting 
candidates into principal preparation programs. Sarason (1999) argues that selecting only- 
on the basis of conventional objective data is not justifiable on moral and educational 
grounds. He continues by making an analogy to the performing arts: “if you want to 
predict who will make a good actor, you have to see them act, keeping in mind that you 
are observing an amateur” (p. 99). 
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Table 2. Irate Parent Call Assessment Form 



Student Identification Number A-1 












Communication look-fors 


1 


2 


3 


4 


5 



conveys ideas and opinions succinctly 
expresses clear and concise language 
checks for understanding 
uses appropriate language 
exhibits sensitivity to parent concerns 
renders a timely and appropriate decision 
displays appropriate listening skills 
ADDITIONAL COMMENTS: 

1 . Appeared in a hurry to get the parent off the phone. 

2. Be careful about putting the responsibility on the parent to check these 
issues out - that’s why she’s calling you. 

3. Be more explicit about what steps you will take. 

4. Strive to show empathy to the parent and at the same time be 

supportive of the teacher. 

Note: The administrator constructs and descriptors listed above are to be used as “look- 
fors” as the audition activity unfolds. Additional comments will help with the assessment 
of the individual. Observation should be rated on a scale of 1-5, with 1 indicating little 
evidence and 5 representing strong evidence. 



_X_ 

_X_ 

_x_ 

X 



_x_ 

_x_ 

X 



Audition #2. Concerned Parent Group. Students individually entered a room to 



meet a group of concerned parents, role played by a teacher, principal, and university 
professor. The student received the following script approximately 30 minutes prior to 
the meeting: 

A concerned parent group wants to meet with you. This watchdog group 
wants to know how well your eighth-grade students have done on the 
statewide standardized test in reading, written language, and mathematics 
for each of the past three years. The group wants to know what the numbers 
are and how to interpret them. You are to make a brief presentation (5 minutes) 
to this group. The test results for the last three years are shown in Table 3. 

Table 3 Eight-Grade Test Results 

Rank of School Statewide 
Eight Grade 

(Expressed in Percentiles) 



Year 


i 


2 


3 


Reading 


80 


80 


70 


Written Language 


90 


60 


50 


Mathematics 


90 


70 


50 



As with audition #1, the issue here is not a perfect performance necessarily, but 
whether or not the candidate possesses the important qualities required in the handling of 
a concerned parent group. In many cases, students incorrectly explained the meaning of 
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percentile ranks and stated, “the 50* percentile means your son answered half the 
questions correct.” Some excused the decline in scores because of weak teachers and 
others stated that there had been an increase in Hispanic and African American students 
over the last several years. An assessment form of an especially low-performing 
candidate is displayed in Table 4. 

Table 4. Concerned Parent Group Assessment Form 



Student Identification Number B-3 


Communication look-fors 


1 


2 3 4 5 


adequately explains data 


_x_ 


— — — — 


exhibits sensitivity to parent concerns 


— 


_x_ _ _ _ 


accepts responsibility for school data 


_x_ 


— — — — 


checks for understanding 


— 


_x_ _ _ _ 


uses appropriate language 


— 


_x_ _ _ _ 


displays appropriate listening skills 




_x_ _ _ _ 



ADDITIONAL COMMENTS: 

1. Did not accept responsibility - made the situation worse by saying that 
the district hired several weak teachers over the last three years. 

2. Said to the parents that test scores were not that “big of a deal.” 

3. Talked all the time - did not ask if the parents understood. 

4. Made light of the situation and talked down to the group. 
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The performance of this particular aspiring principal failed to display evidence of 
qualities of effective leadership such as: (a) exhibiting sensitivity to parent concerns, (b) 
recognizing the importance of parent issues, and (c) displaying appropriate listening 
skills. Certainly, some of these qualities can be nurtured and improved upon during the 
educational administration preparation program. However, one must suspect that this 
individual lacks many of the basic skills, knowledge, and dispositions necessary in 
effective school leadership. The candidate actually went on to graduate from the program 
but to the date of this writing has not secured a position in educational administration. As 
the former coordinator of educational administration programs at this candidate’s 
university, I am troubled still as to why we did not make a decision of rejection in this 
case. In retrospect, I believe we made a professional, moral, and ethical mistake in 
accepting a candidate with such “borderline characteristics and qualities” of effective 
leadership, not to mention a disservice to the individual. 

Conclusion 

We must discontinue the practice of admitting candidates to leadership 
preparation programs on the sole basis of the variety of non-behavior based measures 
such as test scores, interviews, and letters of recommendation. These measures do not 
give us an estimate of how the candidate will interact with individuals and groups 
encountered on a daily basis at the school site. The audition process can help us assess 
candidates in situations calling for “inventiveness, spontaneity, and sensitivity” (Sarason, 
1999, p. 113). 

Again, the suggestion is not to ignore grade point averages, recommendations, 
and Graduate Record Examination scores in the selection process. The measures, though 
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of limited value, give us important information on the candidate. But the audition gives 
us an additional assessment of the candidate through a more valid behavior-based 
measure. Creating audition situations along with valid and reliable ways of assessing 
them is time consuming, more expensive than traditional methods, and involves a 
commitment from faculties, departments, and colleges of education. 

In discussing the selection of today’s teachers, Seymour Sarason (1999) states the 
following: 

I should hasten to add that the selection of teachers is by no means the 
only major problem in our educational system. But it is a major problem, 
and to deny that is to indulge ignorance, to avoid accountability, and to 
continue to short change future generations of students and teachers, (p.l 13) 

I urge you to substitute the word administrator for the word teacher in the above. It is 
appropriate to consider whether the audition added to the traditional selection criteria can 
improve the quality of perspective educational leaders and in turn practicing principals 
and superintendents. 

The temptation to de -emphasize recruitment and selection is done at the risk of 
compromising educational administration program quality, and to assume that the correct 
mix of high quality candidates will be available to fill the huge number of anticipated 
vacancies in the next decade is a terrible mistake. Acceptance into school administration 
preparation programs should indicate more than perseverance and one’s willingness to 
complete the courses. It should indicate that some of our best, most well-prepared, and 
most creative people have entered the field. The time has come to take a more rigorous 
approach to the selection and training of school administrators. 
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